Watching You Watch Movies: Using Eye Tracking to Inform Cognitive Film Theory

Smith 2 "The art of plastic composition consists in leading the spectator's attention through the exact path and with the exact sequence prescribed by the author of the composition. This applies to the eye's movement over the surface of a canvas if the composition is expressed in painting, or over the surface of the screen if we are dealing with a film-frame." One of the key intentions of cognitive film theory is to understand the cognitive processes involved in the viewing of a film and how this relates to the intentions and decisions of a filmmaker. Film theorists have applied an array of intellectual approaches to this question including the psychoanalytical, philosophical, close formal analyses of films and the application of cultural and social theories (Bordwell & Carroll, 1996; pg. 444). These theoretical explorations have generated many rich and detailed hypotheses about how filmmaker decisions may influence viewers but these hypotheses generally remain untested. In order to identify evidence in support of or in disagreement with these hypotheses, cognitive film theorists can appeal to the methods and theories of empirical psychology and the allied disciplines of cognitive science and cognitive neuroscience. In order for empirical psychology methods to be applied to questions of cognitive film theory a framework is required that demonstrates how questions in one discipline can be mapped into another. In this chapter the Cognitive Computation Cinematics (CCC) approach will be presented. The CCC approach triangulates our understanding of how we watch films via three traditionally separate approaches: 1) cognitive psychology and associated methods of hypotheses testing; 2) computational methods in audiovisual analysis and computational modeling; and 3) the formal and statistical analysis of film (Cinematics; Salt, 2009). These methods can be combined to varying degrees depending on the question being investigated. Smith 3 For example, in order to understand why certain types of cuts are more "invisible" to viewers than others a CCC approach may be to first run an empirical study: viewers could be instructed to detect cuts during a movie, reaction times measured and compared across different types of cuts (e.g. Smith & Henderson, 2008). Differences between detection rates could then be investigated by recording viewer eye movements across each cut and examining how primitive audiovisual features, such as motion and loudness can account for eye movements across the cut using computational methods (see Mital, Smith, Hill, & Henderson, 2011). Finally, the evolution of such …